Methods and systems for analyzing a field

ABSTRACT

Methods and systems for analyzing a field. The methods and systems acquire a thermal image indicative of thermal energy emitted by the soil and/or plants in the field and process the thermal image to assess variations in certain characteristics of the soil and/or plants.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation patent application of co-pending U.S. patentapplication Ser. No. 15/005,612, filed Jan. 25, 2016, which claims thebenefit of U.S. Provisional Application No. 62/107,120, filed Jan. 23,2015. In addition, this application is related to U.S. Pat. No.9,354,216. The contents of these prior applications are incorporatedherein by reference.

BACKGROUND OF THE INVENTION

The present invention generally relates to methods of monitoring thehealth and growth of plants. More particularly, this invention relatesto aerial imaging of vegetation to determine, monitor, and predict planthealth and growth.

Various technologies have been used in the past to measure temperatureof plant leaves. For example, U.S. Pat. No. 7,058,197 uses visual lightreflectance to generate NDVI (normalized difference vegetation index)images. This patent relies on reflected light from the sun, andtherefore teaches that the optimum time for image acquisition using thedisclosed process is within two hours of “solar noon” and on cloudlessdays. This makes it very impractical for a commercial application. Inparticular, this patent discloses

-   -   Aerial imagery was collected four times throughout the growing        season. The image dates correlated with bare soil, VI2, VT, and        R4 crop stages (see section on “Resolutions in Remote Sensing”).        The aerial imagery was flown with digital cameras with an array        size of approximately 1500 pixels wide and 1000 pixels in the        along track dimension. The digital systems were 8-bit systems        and were collected and stored on an on-board computer in a        Tagged Image Format (TIF). Four bands were collected        representing the blue, green, red, and near infrared portions of        the electromagnetic spectrum (see section on “Spectral Nature of        Remote Sensing”). The cameras were aligned in a two-by-two        matrix and were rigid mounted (pseudo-bore sited) with the        lenses focussed [sic] on infinity. The imagery was flown at        approximately 5000 feet above ground level (AGL) to produce a        spatial resolution of approximately one meter by one meter (see        section on “Resolutions in Remote Sensing”). The digital cameras        have square pixels and are not interlaced during image        acquisition. The optimum time for image acquisition was two        hours before or two hours after solar noon (see section on        “Resolutions in Remote Sensing”). Images were not acquired        during times of poor atmospheric conditions (haze, rain,        clouds). No cloud shadows were acceptable in the imagery.

In addition, it appears that the methodology disclosed by U.S. Pat. No.7,058,197 is only able to indicate that a problem exists after a planthas actually changed its structure, as indicated by its color. In manycases, this is too late to take corrective action. Column 6 of U.S. Pat.No. 7,058,197 describes the extent of the methodology's capability asfollows:

-   -   The third major division of the electromagnetic spectrum ranges        from around 1500 nanometers to approximately 3000 nanometers and        is referred to as the middle-infrared. It is this portion of the        electromagnetic spectrum where moisture plays a dominant role.        Although other factors such as organic matter, iron content, and        clay content have an effect, moisture appears be the primary        mechanism affecting reflectance. More specifically, the higher        the moisture content, the lower the reflectance. As objects lose        moisture or begin to dry, their reflectance in this portion of        the electromagnetic spectrum increases. While this concept has        been proven in a laboratory setting, applying this concept in        practice has been somewhat evasive.

As another example, U.S. Pat. No. 6,597,991 uses thermal imaging todetect water content in leaves for irrigation purposes. This patent isreliant on obtaining actual temperatures and using ground-basedreferences for calibration. Arguably, a significant disadvantage of U.S.Pat. No. 6,597,991 is its reliance on extremely accurate temperaturemeasurements so that the need for irrigation can be determined. Such arequirement necessitates an extra step and additional costs associatedwith the calibration. U.S. Pat. No. 6,597,991 does not appear to containa reference to the detection of disease in very early stages.

U.S. Pat. No. 6,212,824 uses various remote sensing and image analysistechnologies for classifying plants in small fields. In particular, thispatent discloses

-   -   The present invention employs remote sensing technology to        classify inbred and hybrid plants and segregating populations        for commercially important traits such as yield, environmental        stress responses, disease resistance, insect and herbicide        resistance, and drought resistance. Images are prepared from        remote sensing data obtained from plants.    -   These evaluations are useful in decision making to select plants        from early generations or preliminary tests used in breeding, to        be advanced for selective breeding.

U.S. Pat. No. 6,212,824 discloses the use of both thermal imaging andreflectance at various wavelengths (multiple bands) for imagingvegetation from an aircraft in order to classify plants. However, itdoes not appear that the patent uses long-wave thermal images of a typecapable of use for monitoring the growth of vegetation, predictingfuture growth of vegetation, and/or detecting disease, insectinfestation, or other stress factors in vegetation before they becomeapparent to visual or near-infrared cameras. Rather the patent appearsto focus on visual and near-infrared wavelengths. For example, thepatent states

-   -   In an exemplary embodiment, CIR photographs revealed qualitative        differences between the four row subplots across both limited        and full irrigation treatments (FIG. 4). These photographs were        subsequently processed (FIG. 5) to generate quantified values        for the three bands or wavelengths of reflectance used to create        the photograph. The three bands 20 were green, red and        near-infrared (not thermal) portions of the energy spectrum.        FIGS. 6-8 show the green, red and near-infrared results on the        same field as in FIG. 4. The red and near-infrared bands were        considered to be indications of the crop conditions, having been        used by others in crop assessment programs. The red band        corresponds to chlorophyll absorption and, according to theory,        reflectance in this band increases during times of stress.        Reflectance in the near-infrared region is predicted to decrease        with increasing stress. The near-infrared region is believed to        be related to plant structure and composition.

In view of the above, it can be appreciated that there are certainproblems, shortcomings or disadvantages associated with the prior art,and that it would be desirable if an improved method were available foraerial monitoring of plant health and growth that does not rely solelyon sensing reflected visible light and/or ground-based measurements.

BRIEF DESCRIPTION OF THE INVENTION

The present invention provides methods and systems suitable for aerialimaging of soil and/or vegetation in a field to determine, monitor,and/or predict characteristics of the soil and/or plants, and do notrely solely on sensing reflected visible light and/or ground-basedmeasurements.

According to certain aspects of the invention, methods and systems areprovided for assessing variations in temperatures among plants in afield to detect disease in the plants, detect insect infestation in theplants, and/or to predict a future yield of the plants. The methods andsystems use a thermal imaging device in an aircraft to acquire an aerialthermal image indicative of thermal energy emitted by the plants, andprocess the thermal image to assess temperature variations of theplants.

According to other aspects of the invention, methods and systems areprovided for assessing variations in moisture in a field. The methodsand systems use a thermal imaging device in an aircraft to acquire anaerial thermal image indicative of thermal energy emitted by the soil.Moisture content is then sensed at a location in the field, the locationof the sensed moisture content is identified on the thermal image, andthe thermal image is processed to assess moisture variations of the soilin the field.

A technical effect of acquiring and processing thermal images of plantsand soils obtained by aerial imaging is the ability to view an entirefield so that relative differences over areas of the field may becomparatively analyzed.

By acquiring a thermal image indicative of thermal energy emitted byplants or soil and processing the thermal image to assess variations intemperature, a trained thermographer or computer software can monitorconditions that affect growth of vegetation, predict future growth ofvegetation, and/or detect disease, insect infestation, and/or otherstress factors before they become apparent to visual or near-infraredcameras.

Other aspects of the invention include further acquiring a digitalvisual image of at least a portion of a field that is indicative oflight reflected by plants within the portion while the aircraft is inflight, wherein the visual image captures light and is processed toassess relative variations in light reflectance among the plants.

A technical effect of acquiring and processing aerial thermal anddigital visual images is the ability to analyze a field for conditionsthat affect plant health and growth. As a particular but nonlimitingexample, it is believed that, by acquiring and processing a digitalvisual image to assess relative variations in light reflectance amongthe plants and acquiring and processing a thermal image to assessrelative variations in plant temperatures across the field, stressfactors may be identified and used to build prescription maps for seedand fertilizer for individual fields, and to apply rescue treatmentsduring the current season. The imagery may also be used to plan oradjust the planting and fertilizing for the following season.

Other aspects and advantages of this invention will be betterappreciated from the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic plan representing an imaging system in accordancewith an aspect of this invention.

FIG. 2 is a screen shot of a computer software interface providingaccess to a database of images sorted for an individual farm inaccordance with an aspect of the present invention.

FIGS. 3-5 are screen shots of a computer software interface representinga process of aligning and superimposing a thermal image of a field on adigital image of the field to create an overlaid image that is mapped toa geographical map.

FIG. 6 includes an image of a field before and after analyzing the imageto identify differences in color across the canopy of a crop in thefield.

FIG. 7 represents a first thermal image (left) and a second thermalimage (right) illustrating the spread of disease in vegetation in afield. Portions of the field inflicted by disease are identified witharrows.

FIGS. 8-10 represent thermal and visual images corresponding to insectinfestation (FIG. 8) and disease (FIGS. 9 and 10) in vegetation in afield.

FIG. 11 is an exemplary SSURGO map available from the U.S. Department ofAgriculture (USDA).

FIG. 12 includes a digital visual image and a thermal image of a barefield comprising little or no vegetation.

FIG. 13 is an image of a field representing the relative moisturethroughout the field.

FIG. 14 is a screen shot of a computer software interface suitable forremotely inspecting a field with an unmanned aerial vehicle.

DETAILED DESCRIPTION OF THE INVENTION

The present invention is generally applicable to imaging of soil and/orplants in a field utilizing energy (heat) emitted thereby and/or broadspectrum light reflected thereby. A particular aspect of the inventionis based on a determination that by using certain microbolometertechnology in a thermal imaging device, a trained thermographer orcomputer software can monitor the growth of vegetation, predict futuregrowth of vegetation, and/or detect disease, insect infestation, orother stress factors in vegetation before they become apparent to visualor near-infrared cameras. While the invention is described as beingsuitable for monitoring plant health and growth, other applications areforeseeable and therefore the present invention should not be limited tothe described embodiments herein. For example, an aspect of theinvention includes predicting a future yield of a crop based on thermalimages. The nonlimiting aspects of the invention described hereinafterare in reference to obtaining information about a field by imaging atleast a portion of the field while an aircraft 10 is in flight over thefield as represented in FIG. 1, preferably at an altitude of at least5000 feet (about 1500 m) above ground level over the field being imaged.While it is foreseeable and within the scope of the invention thatcertain aspects of the invention may be used at lower altitudes, aminimum altitude is preferred such that entire fields may be viewed andrelative differences over areas of the fields may be comparativelyanalyzed.

While not wishing to be held to any one theory, investigations leadingto certain aspects of the present invention regarding disease detectionin plants indicated that during the day, sick plants tend to be at anincreased temperature relative to healthy plants due to a lack ofevapotranspiration. These sick plants are represented in thermal imagesas being warmer than other healthier plants in the field. In contrast,during the night sick plants tend to be at a decreased temperaturerelative to healthier plants primarily since the healthier plants retainwater comprising latent heat. Therefore, at night sick plants arerepresented in thermal images as being cooler than healthier plants inthe field. The difference in temperature between the healthier plantsand the sick plants will be dependent, at least in part, on theprogression of the disease in the sick plants. Preferably, systems asdescribed herein are capable of detecting variations in temperature thatare relatively small and indicative of disease in a plant at a time inthe progression of the disease prior to the disease being detectable tovisual or near-infrared inspection.

Preferred embodiments of the invention employ a system comprising one ormore high-resolution long-wave thermal imaging cameras 12 thatpreferably can be mounted in the aircraft 10, as schematicallyrepresented in FIG. 1, optionally along with a digital camera 14, forexample, a twenty-one megapixel digital camera, that can be used toprovide a digital image for reference purposes. While the thermalimaging camera 12 and the digital camera 14 may be mounted to theaircraft 10 by any means, preferably the thermal imaging camera 12 andthe digital camera 14 are mounted in a baggage compartment of theaircraft 10 and exposed through several holes in the aircraft skinbeneath the baggage compartment in accordance with FAA AC43.13. Computerequipment 16 for controlling the thermal imaging camera 12 and thedigital camera 14 may be located in the cockpit including a monitor 18for the purpose of displaying and monitoring the thermal and/or digitalcamera images. However, alternative locations for the thermal imagingcamera 12, the digital camera 14, and computer equipment 16 areforeseeable.

A particular by nonlimiting example of a thermal imaging camera 12 thathas been employed with the invention is manufactured by InfraredCameras, Inc., of Beaumont, Tex. USA. The particular model of thisthermal imaging camera is ICI 7640, equipped with a specially made 15 mmlens. The thermal imaging camera 12 produced an array of 480×640 pixels(currently considered to be high resolution, compared to thermal imagingcameras that offer resolutions of, for example, 320×240 pixels and160×120 pixels) and used a microbolometer sensor that changestemperature as a result of being exposed to infrared (IR) energy(wavelengths between 0.7 and 300 micrometers). Images with resolutionslower than 480×640 pixels were found to be blurred when taken from highaltitudes (e.g., at least 5000 feet above ground level over a fieldbeing imaged). Microbolometer sensors are considered to be “long wave”sensors because they collect light in wavelengths much longer thanvisible light, i.e., above the 0.4 to 0.7 micrometer spectral band.Wavelengths above visible light provide better penetration throughsmoke, smog, dust, and other interference that may be present under pooratmospheric conditions.

Suitable microbolometer sensors preferably collect light in wavelengthsof between about 7 and 14 micrometers. The microbolometer sensor thatwas utilized with the ICI 7640 camera is especially sensitive to thermalenergy (infrared radiation) and was capable of sensing wavelengths aslong as 14 micrometers and as short as 7 micrometers or below.Wavelengths below 8 micrometers were found to be especially importantfor identifying disease in plants. During investigations leading to thepresent invention, it was determined that the size of the capturedwavelength ranges influenced the results obtained. For example, widercaptured wavelength ranges (for example, 7-14 micrometers) providedimproved measurements for relative temperatures between plants butreduced accuracy of specific temperature measurements of individualplants. In contrast, narrower captured wavelength ranges (for example,8-13 micrometers) provided improved accuracy of specific temperaturemeasurements of individual plants but reduced measurements for relativetemperatures between plants. It was further determined that measuringrelative temperatures between plants in the field was critical todetecting and predicting disease, insect infestation, or other stressfactors in the plants. Consequently, the microbolometer sensor ispreferably capable of sensing wavelengths below 8 micrometers, and morepreferably at or below 7 micrometers, and preferably simultaneouslycapable of sensing wavelengths above 13 micrometers, and more preferablyat or above 14 micrometers. Equipped with its sensor, the thermalimaging cameras 12 preferably utilized by the invention do not requirereflected light from the sun, and therefore allow the system and methodof this invention to be used under poor atmospheric conditions (haze,rain, clouds, etc.), and even in the darkness of night.

In contrast, U.S. Pat. No. 7,058,197 uses digital cameras that workprimarily with reflected light in wavelengths of 0.38 to 0.72 micrometerto see plant color. These wavelengths are in the visible spectrum andrequire a light source from something above 6000° C., such as a lightbulb filament or the sun. The white light from this energy source thenbounces back (reflects) off objects in different wavelengths, enablingcolors to be seen. Consequently, U.S. Pat. No. 7,058,197 teaches thatthe optimum time for image acquisition is two hours before or two hoursafter solar noon. In contrast, suitable cameras for use with the presentinvention do not require a source of high energy because they measureenergy that is emitted, not reflected, by plants.

U.S. Pat. No. 6,212,824 uses visual wavelengths in the 0.4 to 0.7micrometer range and near-infrared wavelengths in the 0.7 to 1.1micrometer range. These ranges are well outside the 7 to 14 micrometerwavelength range that is believed to be necessary for monitoring thegrowth of vegetation, predicting future growth of vegetation, and/ordetecting disease, insect infestation, or other stress factors invegetation in accordance with aspects of the present invention.

Because thermal imaging cameras believed to be suitable for use withthis invention detect energy waves of much longer wavelengths, (e.g., 7to 14 micrometers), the thermal imaging camera 12 is able to detectobjects over a range of temperatures, for example, about −35° C. toabout 200° C. A computer program can be used to focus the thermalimaging camera's sensitivity onto an area that encompasses a range oftemperatures above and below the ambient temperature of the crop canopy,for example, about 10° C. above and below. A color palette can then beused in the computer program to build an image showing the relativetemperature of the canopy. Such a computer program is well within thecapabilities of those skilled in the relevant art, and therefore willnot be discussed in any detail here.

In the above manner, certain aspects of the invention can be performedto evaluate plants under poor atmospheric conditions and even in totaldarkness and measure plant health and growth based on temperature,without the need for using reflected light as proposed by U.S. Pat. No.7,058,197. A system as described herein can employ a technique by whichthe computer program is written to be further able to compensate forclouds if the images are taken during daylight hours. This can beaccomplished by utilizing the aforementioned separate digital camera 14,whose digital images can overlay thermal images acquired with thethermal imaging camera 12. Since clouds are readily apparent in thevisual digital image of the digital camera 14, the computer program canbe used to compensate for cooler areas that exist beneath clouds. Forthis purpose, both thermal and visual digital images are preferablyacquired of the field of interest simultaneously. The computer programmay then analyze the visual digital image to recognize which portions ofthe image were under cloud cover, and then may adjust the correspondingportions in the thermal image to compensate for the cloud coverage.Similar methods may be used to compensate for other interfering elementssuch as wind.

At the time of the invention, thermal imaging cameras 12 of the typeused by the invention were believed to be limited to one camera mountedon NASA's Space Shuttle and another leased to the University ofWashington for environmental research. Up until the time of theinvention, it was believed that there were no uncooled microbolometerthermal imaging cameras 12 commercially available with resolutionssuitable for use with the present invention. This camera technology waspreviously developed for the U.S. military as the heat-seeking elementin missile guidance systems, and has recently been made available to thegeneral public. While thermal imaging cameras have been available formany years prior to the time of the invention, they were required to becooled with liquid nitrogen and therefor impractical for use in mostaircraft for safety and economic reasons.

Through the use of thermal imaging cameras of the types described above-and analytical software, a trained thermographer or software program canidentify very subtle relative differences in canopy temperature. Infact, the thermal imaging camera manufactured by Infrared Cameras, Inc.,as modified herein is capable of measuring differences of as little as0.03° C. Through careful analysis of the thermal images, these subtledifferences in temperature can guide a user on the ground to suspectareas in a field. For example, analysis of the thermal images may revealpatterns that stand out from those normally observed in healthy plantsand may be related to patches of diseased or insect infested plants.FIGS. 8 through 10 represent exemplary thermal images comprising insectinfestation and/or disease in vegetation in a field as well as visualimages of the afflicted vegetation identified in the thermal images. Inparticular, FIG. 8 represents a thermal image of a field of alfalfacomprising a region where crops infested with pests(leafhoppers/Cicadellidae) were found, FIG. 9 represents a thermal imageof a field of corn comprising a region where diseased (gray leaf spot)crops were found, and FIG. 10 represents a thermal image of a field ofsoybeans comprising a region where diseased (white mold) crops werefound. Preferably, the system includes a computer software programcapable of identifying patterns in the thermal images indicative ofdiseased plants. In order to detect disease or other ailments in time totreat the condition, and in particular detect such ailments at a time inthe progression of the ailment prior to the ailment being detectible tovisual or near-infrared inspection, the thermal imaging camera 12preferably is extremely sensitive to relative temperatures betweenindividual objects. Preferably, the thermal imaging camera'ssensitivity, generally measured by a parameter referred to as noiseequivalent temperature difference (NETD), is at least 0.03° C. or lowerat optimum ambient temperatures. Less sensitive cameras have been foundto be incapable of properly identifying differences in temperaturenecessary for early detection of disease in vegetation in fields imagedfrom high altitudes (i.e., greater than 5000 feet above ground level).

Monitoring the health and growth of the plants may be further improvedby acquiring a plurality of thermal images over a period of time, forexample acquiring at least one thermal image of a field every two weeksduring a growing season. Preferably, a computer program may then analyzeor overlay multiple thermal images of a single field to assign a valueto each pixel in the images. Patterns of disease or insect infestationmay then be recognized and assessed. For example, disease may bespreading among the plants if the sizes of the patterns are increasing,or the plants may be recovering if the sizes of the patterns aredecreasing. Preferably, the computer program applies a color palette tobring the identified pattern to the attention of an observer such as thefarmer or agronomist. FIG. 7 represents a first thermal image (left) anda second thermal image (right). The second thermal image was taken tendays after the first thermal image and represents increases in threedifferent diseased areas in an imaged field, each area indicated with anarrow.

In investigations leading to the present invention, it was determinedthat the difference in temperature (delta) between a lowest temperatureof the plants in a given field and a highest temperature of the plantsin the same field correlates directly with a yield potential of thatfield. For example, in low yield years the delta in temperature has beenfound to be fairly large due to the drastic contrast between the damagedplants and the plants in less affected areas of the field. In contrast,in high yield years where there has been less stress on the plants, thedelta has been observed to be generally much smaller. In view of thesefindings, the software preferably may be used to analyze the thermalimage and perform a yield prediction for at least part of a field basedon variations in temperature, optionally along with farmer commentsand/or other variables. According to one aspect of the invention, thesoftware may provide a yield estimating tool that uses the thermalimagery to assign each pixel of a thermal image within a field boundarya yield prediction based on temperature, and user input at designatedcheck points. For example, a farmer may physically perform yield checksat three locations in the field corresponding to three pixels on thethermal image having a coolest temperature, a warmest temperature, and atemperature therebetween and possibly midway between the coolest andwarmest temperatures, and the software may then compare this informationto the rest of the pixels in the field to perform a yield prediction ofthe entire field by comparing the pixels at the checked locations toother pixels in the thermal image. Since the yield prediction is basedon each individual pixel in the captured thermal image, the predictioncan be substantially more accurate than conventional yield estimatemethods such as counting the yield in a random area of the field andthen extrapolating this count to predict a yield of the entire field. Inparticular, yield predictions according to aspects of the presentinvention preferably take into account diseased areas of the field andthe size of the produce grown. For example, when predicting the yield ofa corn field, the yield prediction algorithm preferably takes intoaccount the health of the plants, the size of the ears of corn that maybe produced on the plants, the number of kernels on each ear of corn,and other specific information relative to each pixel in the thermalimage which may not normally be accounted for in conventionalpredictions. Although the above nonlimiting process requires a fieldcheck of a minimum of three locations in the field in order to predictyield, additional locations may be field checked in order to potentiallyprovide more accurate yield prediction results. Alternatively, it isforeseeable that the yield prediction may be calculated based solely onthe difference in temperature between the warmest plants in the fieldand the coolest plants in the field, which may then be analyzed by thesoftware in comparison to data accessible by the software, for example,known temperatures and yields of previous years and/or genericprediction algorithms of the software.

It is foreseeable and within the scope of the invention that the yieldprediction process described herein may have broader applications thanpredicting the yield of a single field. In particular, a plurality ofthermal images of a plurality of fields may be obtained and used to inaggregate to create a broad area yield index, representing a yieldprediction for a geographic area corresponding to a region, state,country, or other relatively large coverage area. For example, adatabase may be created containing thermal images of fields across anentire country. The yield prediction process as previously described maybe performed on each individual field. By combing the results of all ofthe fields in the database, an accurate national yield index may becalculated for the country. Alternatively, the national index may beproduced by inspecting various points among the imaged fields to provideyield related data to the system. These points may then be compared tothe rest of the pixels in the plurality of thermal images to calculatethe yield prediction in the same manner as would be done for analyzing asingle field. The thermal images used for determining the yield indexmay be filtered to provide indexes specific to individual crops, plants,or other parameters of interest. For example, the thermal images may befiltered such that only corn fields are represented which then providesa corn yield index for the coverage area. It is believed that yieldindexes as described herein would be available sooner and are moreaccurate than conventional analyst publications, such as but not limitedto the crop yield indexes published by the U.S. Department ofAgriculture (USDA).

According to one aspect of the present invention, the digital visualimage of the digital camera 14 may be analyzed and/or processed in orderto render a hypersensitive “green filtered” image or Advanced DigitalVegetation Index (ADVI) image. For this purpose, the digital visualimage preferably captures light in the near-infrared, visual, andultraviolet spectrums. For this purpose, the digital camera 14preferably captures as much of the light spectrum as possible. Ininvestigations leading to aspects of the present invention, it wasdetermined that by capturing images with commercially available high enddigital cameras, and further using a computer algorithm to infer lightintensity slightly beyond the range captured by the camera, it waspossible to obtain a wavelength range suitable for producing the ADVIimages. By using a combination of near-infrared light coupled with redabsorption, green reflectance, and portions of the ultraviolet spectrum,the software may assign each pixel of the digital image an intensityvalue, for example, 1 to 255. The software may then apply a colorpalette to the image based on the pixel intensity values. The resultingADVI image may represent differences in color across the crop canopy.FIG. 6 represents an aerial image of a field (left) and an ADVI image ofthe field (right) in accordance with an aspect of the invention. SuchADVI images may be analyzed in order to determine information such asbut not limited to nitrogen loss in plants within a field. For example,by comparing an ADVI image and a thermal image of a field, a trainedthermographer or computer software can determine if abnormalities to agroup of plants are due to disease or lack of nutrients. Thisinformation may be used to build prescription maps for seed andfertilizer for individual fields, and to apply rescue treatments duringthe current season. These rescue treatments may include treating onlythe detrimentally effected plants in the field to address the disease orlack of nutrients. The imagery may also be used to plan or adjust theplanting and fertilizing for the following season.

Although U.S. Pat. No. 7,058,197 discloses creating NDVI images based onlight reflectance, only wavelengths in the visible spectrum between 0.38to 0.72 micrometers are disclosed as being captured and analyzed. Incontrast, the ADVI images disclosed herein are created by capturingdigital images that capture as much of the light spectrum as possible,including near-infrared, visual, and portions of the ultravioletspectrum and then applying computer algorithms to the digital images todetermine the relative light reflectance between plants in the field. Assuch, the present ADVI images take into account substantial amounts ofdata which is believed to be excluded from the analysis performed inU.S. Pat. No. 7,058,197.

In addition to analyzing the health and nutrition of plants in a field,the ADVI images may be used to analyze bare fields prior to plants beingplanted in the field in order to determine the types of soil in thefield. Such information can be beneficial for producing prescriptionmaps. Conventionally, the soil type of a field is defined prior todetermining an agricultural prescription. In the United States, this hashistorically been done using SSURGO (soil survey) maps that wereoriginally generated by the USDA in the early 1970's. An exemplarySSURGO map is represented in FIG. 11. These maps are based on subjectiveobservation by humans, and therefore may often be inconsistent and/orinaccurate. Because most farmers currently build their foundations forprescriptions from these maps, errors are inherently embedded in thesefoundations, and erroneous prescriptions are then generated. Byutilizing a combination of the ADVI imagery in conjunction with thermalimagery as disclosed herein, it is possible to derive more accurate soilzones to use for these prescriptions. For example, both a visual digitalimage and a thermal image may be obtained of the bare soil (i.e., littleor no vegetation) of a field. Exemplary images of a bare field arepresented in FIG. 12 (visual image on left, thermal image on right).From the digital visual image, an ADVI image may be produce and thencompared by the software or a trained thermographer to the thermal imageto determine all of the various types of soil present in the field andtheir locations. Notably, the thermal image can be used to compensatefor moisture in the field. For example, if only the ADVI image wasanalyzed without the use of the thermal image, wet areas of the fieldcould be erroneously mistaken for areas of darker soil, because moisturecan have the effect of reducing light reflectance. By comparing the ADVIimage to the thermal image, it is possible to determine that a darkerarea of the field is merely wet, or relates to soil type that is darkerthan surrounding soil types. By analyzing the two types of images, asoil zone map may be produced accurately indicating boundaries or zonesbased on the colors captured within a field. As such, the softwarepreferably includes a soil type difference indicator feature suitablefor producing soil zone maps of fields based on the intensity of lightreflected from the bare soil. Optionally, the software may allow a userthe ability to choose how many zones the field is broken into (e.g., byadjusting the color range per zone). Soil zone maps produced in thismanner will not comprise the errors believed to be inherent inprescription maps produced based on the subjective findings of humanssuch as the SSURGO maps.

According to one aspect of the invention, a thermal image of a fieldobtained as described herein may be used in combination with otherdata-obtaining tools to provide additional information about the field.For example, the software may have an irrigation planning feature thatworks in combination with a soil moisture probe to calculate themoisture throughout an entire field, and may produce a moisture imagerepresentative of such moisture data. Currently, many farmers thatutilize irrigation invest in at least one soil moisture probe for eachof their fields. These probes and associated services are often veryexpensive. In addition, the probes are only capable of determiningmoisture content in the ground at the specific location where it islocated, and not the moisture content of the rest of the field.According to a nonlimiting example, the GPS location of the single probein the field can be overlaid on top of a thermal image of the field(i.e., assigned to a pixel in the thermal image corresponding to thelocation of the probe), and the software can then compare the relativetemperatures of the pixel corresponding to the probe's location andother pixels in the thermal image to determine the moisture content ofeach pixel in the field. Effectively, this process multiplies the probeby millions across the entire field. In particular, the relativity amongthe pixels in the thermal image can be used to determine areas of thefield that are wetter (cooler) than the location of the probe, and areasthat are dryer (hotter). Based on this moisture information, thesoftware or a user may determine the moisture content at any locationwithin the field and how much, if any, water to apply to varioussections of the field. Such information could further be uploaded intoan irrigation controller, such that water may be automatically andselectively applied or not to various sections of the field depending onthe real time requirements of each section. An irrigation systemutilizing this data is believed to provide both economic as well asenvironmental benefits due to improved utilization of water. FIG. 13represents a moisture image representative of moisture in a field.Circular moisture regions are represented that were formed due to theextents of an irrigation system operating in the field. Light coloredrings can be seen within the circular areas representing warmer areascaused, in this instance, by nozzles on the irrigation system that wereplugged and therefore not applying as much water.

Certain aspects of the invention encompass various modifications andimprovements to the system described above, including but not limited toassisting ground inspection of the fields. For example, FIG. 1schematically represents the use of a tracking device 22, such as aG.P.S. (Global Positioning System) device, located within or on theaircraft 10 to log the position of the aircraft 10 on timed intervals,such as once per second, in addition to storing the compass heading ofthe aircraft 10. Alternatively, the position and heading of the aircraft10 may be logged by a system when an image is acquired by any of theimaging devices on the aircraft, such as the thermal imaging camera 12or digital camera 14. Computer software can then be used to synchronizethis information with the time at which each individual image is taken.The software may then rotate and orient all of the images in, forexample, a “north up” orientation for easier referencing during groundinspection regardless of the flight path of the aircraft 10. Forexample, if a thermal image is acquired while the aircraft 10 is flyingsouth, the thermal image will inherently represent the southern portionof the field near the uppermost portion of the image, that is, in a“south up” orientation. The software can utilize the logged position andheading of the aircraft 10 to rotate the image such that the northernportion of the field is in the uppermost portion of the image, that is,a “north up” orientation.

The information recorded by the tracking device 22 may further be usedwith computer software to geo-locate (georeference) all of the images(e.g., thermal images, visual digital images, ADVI images, soil zonemaps, moisture images, etc.) and sort them into groups for eachindividual tract of land, for example a farm. Preferably, the softwaremaps the images by superimposing each image on a geographical map at thecorresponding location where the image was taken. In addition toimproving ground inspections, this data may be used to compile adatabase of images sorted according to each individual farm, forexample, as represented in FIG. 2. Preferably, the database comprisesnames for the fields and their corresponding coordinates. This allowsthe software to not only superimpose the image on a geographical map atits corresponding location but also to provide the name of the fieldcaptured in the image. For example, the image may be superimposed on ageographical map with a digital tag or label comprising the name of theclient, farm, and/or specific field represented in the image. For largerdatabases, the image may be labeled to comprise additional information,such as a name of a company that owns, leases, or is otherwise relatedto the farm, a division within the company, an agronomist, the client,farm, specific field, or other relevant information. Preferably, thesoftware provides tools for modifying the images, correcting for cameraerrors, comparing and magnifying images, exporting images to variousfile formats, adding and saving user or system generated comments to theimages, and notifying the companies, clients, etc., via email, mobiletexting, or the like of changes to their respective accounts, such as anew image being added to the database.

Ground inspections can further be improved by providing the user withaccess to a database of images as described herein during the physicalground inspection. This may be accomplished either by providing ageoreferenced image to a handheld mobile device (for example, a tabletcomputer, cellular phone, laptop computer, or the like), or bygeoreferencing the image with such a mobile device using softwarewritten specifically for this purpose. Preferably, the software isadapted to automatically download each image associated with aparticular parcel or client, for example using a File Transfer Protocol(FTP) method, and allow viewing of all the associated images.Alternatively, if the handheld device has wireless Internetcapabilities, the software may be adapted to communicate remotely withthe database, and allow viewing of the images without the need todownload the images. The user can then choose to load any image to anoverlay map and manually manipulate the image to align field boundaries.

According to one aspect of the invention, the software allows the userto choose two common points between the image and a map in order togeoreference the images. For example, FIG. 3 represents a screenshotwith a geographical map on the left and a thermal image of a field onthe right. FIG. 4 represents two geographical points identified on boththe geographical map and the thermal image by a user. FIG. 5 representsthe thermal image overlaid on the geographical map based on the twocommon geographical points identified by the user in FIG. 4. Preferably,the software is adapted to allow the user to adjust the color tone andtransparency of the image for viewing purposes, for example, to create acomposite image comprising aspects of both the image (e.g., thermalimage, visual digital image, ADVI image, soil zone map, moisture image,etc.) and the underlying map that are visible. The user can then walkthrough the field with the user's present location shown on top of theimage, geographical map, or composite image, allowing the ability toprecisely navigate to the areas of most interest. For this purpose, thehandheld device preferably comprises G.P.S. tracking capabilities orother location sensing functionality. In the case of poor plant health,the user can then gather samples and determine the cause of the anomalyin question. If this invention is used for analyzing a farm, timelinessof the ground inspection of the plants is critical for the farmer tomake important management decisions. Therefore, quick access to theimages is a particularly advantageous aspect of the process and system.

In addition to assisting in ground inspections as described previously,the system and/or images described herein may assist in remoteinspection of a field. For example, it is within the scope of theinvention that a field may be inspected remotely with an unmanned aerialvehicle (drone) or other small aircraft that may be remotely flown(manually or autonomously) over a field that has been imaged inaccordance with embodiments of the present invention. In nonlimitinginvestigations leading to aspects of the present invention, a drone wasmodified such that all of its conventional controls were bypassed. Acommercially available “smart” cellular phone was mounted to a bottomsurface of the drone. While other functionally equivalent componentscould be used in place of the phone, it provided several notablefunctions such as but not limited to visual imaging with a camera,location tracking with a G.P.S. device, computing capabilities, andcommunication means through a cellular network. During operation, thedrone flew over a field while the camera on the phone provided a livecamera view of the field from the bottom of the drone to a userinterface, in this instance a mobile app operating on a computer tablet.A user was able to observe a georeferenced aerial image (for example, athermal image) of the field and indicate thereon locations in the fieldthat the user desired to inspect. The user interface sent a signalcomprising the locations to a server, which then sent the signal to thephone on the drone. The phone then controlled the drone to autonomouslyfly over the specified locations in the field. During flight, the phonewas in constant communication with the user interface, providing thelive camera view and location of the drone to the user interface. At anytime during the flight, the user interface could be used to instruct thephone to capture still images of the field, particularly over thepotentially afflicted areas indicated by the aerial image. The capturedimages were saved in full resolution and sent to a server for storageand further inspection. FIG. 14 represents a screen shot of a userinterface for operating the drone. The interface includes ageoreferenced thermal image (right) with a location 30 indicated(pinned) thereon, and a digital visual image (left) representing acontemporaneous view of the field from the drone as it flew over thefield. The location of the drone at the time the digital visual imagewas acquired is indicated on the thermal image by a solid circle 32. Aparticularly useful benefit of using a smart cellular phone was thedrone could be controlled through the cellular network, rather thanrelying on a Wi-Fi signal or other wireless means of communication thatmay have less coverage compared to commercial cellular networks.

Remote inspection methods such as the above example allows for a fieldto be remotely inspected by a trained agronomist without their actualphysical presence being needed at or near the field. For example, thedrone could be launched at the field remotely, autonomously, or by aperson at the field. A trained agronomist may then take control of thedrone and perform an inspection of the field remotely, regardless of thelocation of the agronomist. This would allow the trained agronomist toremotely inspect fields around the world from a single location.

The importance of the technology provided by the present inventioncannot be overstated due to recent developments in agriculturalpractices. Chemical supply companies have released fungicide productsonto the market to combat many diseases of nearly all commercially growncrops. These fungicides have proven to provide such an economicadvantage that many farmers have preemptively contracted this service tobe sprayed from airplanes using a “blanket coverage” technique, andtherefore even in areas where disease is not present. Though this may beconsidered a preventative measure and potentially beneficial, in manyinstances it may not. With the process of the present invention, crophealth can be monitored to enable a farmer to react in sufficient timeto mitigate damage in the event that a crop becomes infested. This isenormously beneficial from an economic standpoint, and quite possiblyfrom an environmental standpoint. In addition to monitoring planthealth, the thermal images may be used to determine other vitalstatistics regarding the plants. For example, the thermal images may beanalyzed in order to inform the farmer of a predicted yield of a cropover an entire farm or for portions of a single field and/or to identifynitrogen loss in a crop.

Though U.S. Pat. No. 7,058,197 broaches certain topics of interest tothe present invention, the method and system described herein rely onwavelengths in the range of about 7 to 14 micrometers, which is adifferent region of the electromagnetic spectrum. U.S. Pat. No.7,058,197 appears to make an assumption that moisture is the primarymechanism affecting reflectance, more specifically, higher moisturecontents correspond to lower reflectance. Though not wishing to be heldto any particular theory, the present invention recognizes thesignificance of radiated energy, in other words, warmer surfaces emitmore energy, and that moisture is simply a medium that promotes energyabsorption as opposed to energy emittance. It also appears to be evidentthat U.S. Pat. No. 7,058,197 relies on reflected light energy, whereasthe process and system of the present invention do not require light,but simply measure infrared radiation from the target body.

The process of the present invention also does not require a calibrationprocedure of the type required by U.S. Pat. No. 6,597,991, and is moredriven by relativity. Therefore, ground-based measurements are notrequired for the present invention.

While the invention has been described in terms of particular equipmentand technologies, it is apparent that other forms could be adopted byone skilled in the art. For example, improved technologies could providegreater resolution of the thermographic image. Furthermore, it isforeseeable that infrared images could be acquired with satellitecameras, though as yet resolution is believed to be inadequate for usewith the present invention due to atmospheric attenuation. Therefore,the scope of the invention is to be limited only by the followingclaims.

The invention claimed is:
 1. A method of analyzing a field, the methodcomprising: positioning an imaging system in an aircraft, the imagingsystem comprising a long-wave thermal imaging camera operable to collectthermal energy emitted by plants; acquiring with the long-wave thermalimaging camera an aerial thermal image of at least a portion of thefield that contains plants while the aircraft is in flight over thefield, wherein the aerial thermal image comprises pixels indicatingdifferent levels of thermal energy emitted by the plants within theportion of the field while the aircraft is in flight over the field;identifying at least three of the pixels on the aerial thermal image, afirst of the pixels indicating a lowest temperature in the aerialthermal image, a second of the pixels indicating a highest temperaturein the aerial thermal image, and a third of the pixels indicating atemperature between the lowest and highest temperatures of the first andsecond pixels; processing the aerial thermal image to assess relativevariations in temperatures among the plants within the portion of thefield by assessing variations in the pixels of the aerial thermal image,determining a temperature difference between the lowest and highesttemperatures of the plants, and comparing the temperature difference andthe first, second, and third pixels to each of the pixels in the aerialthermal image to assign each of the pixels thereof with a yieldprediction based on temperature to perform a yield prediction of theplants in the portion of the field.
 2. The method of claim 1, furthercomprising: georeferencing the aerial thermal image by superimposing theaerial thermal image on a geographical map of the field to obtain ageoreferenced aerial thermal image; providing the georeferenced aerialthermal image to a handheld device located in the field; and locatingand displaying the present position of the handheld device on thegeoreferenced aerial thermal image.
 3. A system for analyzing a field,the system comprising: an imaging system in an aircraft, the imagingsystem comprising a long-wave thermal imaging camera operable to collectthermal energy emitted by plants and acquire an aerial thermal image ofat least a portion of the field that contains plants while the aircraftis in flight over the field, wherein the aerial thermal image comprisespixels indicating different levels of thermal energy emitted by theplants within the portion of the field while the aircraft is in flightover the field; means for identifying at least three of the pixels onthe aerial thermal image, a first of the pixels indicating a lowesttemperature in the aerial thermal image, a second of the pixelsindicating a highest temperature in the aerial thermal image, and athird of the pixels indicating a temperature between the lowest andhighest temperatures of the first and second pixels; means forprocessing the aerial thermal image to assess relative variations intemperatures among the plants within the portion of the field byassessing variations in the pixels of the aerial thermal image,determining a temperature difference between the lowest and highesttemperatures of the plants, comparing the temperature difference and thefirst, second, and third pixels to each of the pixels in the aerialthermal image to assign each of the pixels thereof with a yieldprediction based on temperature to perform a yield prediction of theplants in the portion of the field.
 4. The system of claim 3, furthercomprising: means for georeferencing the aerial thermal image bysuperimposing the aerial thermal image on a geographical map of thefield to obtain a georeferenced aerial thermal image; and means forproviding the aerial image on the geographical map to a handheld devicelocated in the field and having means for locating and displaying thepresent position of the handheld device on the georeferenced aerialthermal image.